Prerequisites#
Home 3B v3#
This new version is based on StableLM-Zephyr-3B, it achieves a score of 97.11% score for JSON function calling accuracy on the test dataset it’s trained
This version supports blind, light, garage_door, media_player, fan, lock, climate, switch, vacuum, todo, input_select and timer entity types.
And now with multi-language support: English, German, Spanish and French
fixt/home-3b-v3:TAG| Name | TAG | Bits | Size | Max RAM required | Use Case |
|---|---|---|---|---|---|
| Home-3B-v3.q2_k.gguf | Q2_K | 2 | 1.08 GB | 3.50 GB | smallest, significant quality loss - not recommended for most purposes |
| Home-3B-v3.q3_k_m.gguf | Q3_K_M | 3 | 1.39 GB | 3.89 GB | very small, high quality loss |
| Home-3B-v3.q4_k_m.gguf | Q4_K_M | 4 | 1.71 GB | 4.21 GB | medium, balanced quality - recommended |
| Home-3B-v3.q5_k_m.gguf | Q5_K_M | 5 | 1.99 GB | 4.49 GB | large, very low quality loss - recommended |
| Home-3B-v3.q8_0.gguf | Q8_0 | 8 | 2.97 GB | 5.47 GB | very large, extremely low quality loss - not recommended |
Use the button below to go to the Integrations Page
To use this version, click on Configure and on Prompt Format select Zephyr
Home 3B v2#
fixt/home-3b-v2:TAG| Name | TAG | Bits | Size | Max RAM required | Use Case |
|---|---|---|---|---|---|
| home-3b-v2.q2_k.gguf | q2_k | 2 | 1.11 GB | 3.67 GB | smallest, significant quality loss - not recommended for most purposes |
| home-3b-v2.q3_k_m.gguf | q3_k_m | 3 | 1.43 GB | 3.98 GB | very small, high quality loss |
| home-3b-v2.q4_k_m.gguf | q4_k_m | 4 | 1.74 GB | 4.29 GB | medium, balanced quality - recommended |
| home-3b-v2.q5_k_m.gguf | q5_k_m | 5 | 2.00 GB | 4.57 GB | large, very low quality loss - recommended |
| home-3b-v2.q8_0.gguf | q8_0 | 8 | 2.96 GB | 5.46 GB | very large, extremely low quality loss - not recommended |
The above RAM figures assume no GPU offloading. If layers are offloaded to the GPU, this will reduce RAM usage and use VRAM instead.
Activate WSL Server Mode#
WSLPorts is a PowerShell script that allows you to easily redirect and open ports in the Windows Firewall for your WSL Services.
Save it on C:\WSLPorts\WSLPorts.ps1
Automate Script Execution at Logon#
Automate Script Execution at Logon
$a = New-ScheduledTaskAction -Execute "powershell.exe" -Argument "-ExecutionPolicy Bypass -WindowStyle Hidden -File `"C:\WSLPorts\WSLPorts.ps1`" "
$t = New-ScheduledTaskTrigger -AtLogon
$s = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries
$p = New-ScheduledTaskPrincipal -GroupId "BUILTIN\Administrators" -RunLevel Highest
Register-ScheduledTask -TaskName "WSLPorts" -Action $a -Trigger $t -Settings $s -Principal $pBased on this article
Llama Conversation Integration#
By acon96
v0.2.5 is out
If you installed v0.2.4, you need to set max tokens to 128 (or your preferred value) in the configuration after the update
Remember to update#
to update it
docker compose pull
docker compose up --force-recreate -d